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/  ABSTRACT 

^  Monte  Carlo  approximation  is  a  useful  method  in  obtaining  a  numerical  approximation  to 
a  Bayesian  action  (an  action  which  minimizes  the  posterior  expected  loss).  We  study  the 
behavior  of  the  Monte  Carlo  approximation  when  the  Monte  Carlo  sample  size  is  large.  Con¬ 
vergence  and  convergence  rate  of  the  Monte  Carlo  approximation  are  established  under  some 
weak  conditions  on  the  loss  function.  )  cr - 
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1.  Introduction 

Monte  Carlo  integration  (Hammeisley  and  Handscomb,  1964)  is  a  very  useful  method  for 
numerical  calculation  in  Bayesian  analysis  when  the  Bayesian  solution  (action)  of  the  problem 
can  not  be  obtained  analytically.  Unlike  other  numerical  methods,  the  use  of  Monte  Carlo 
method  does  not  require  restrictive  conditions  such  as  the  dimension  of  the  parameter  space  is 
low  (say  one  or  two)  and  the  total  number  of  sample  observations  is  large.  Applications  of  this 
method  in  Bayesian  analysis  can  be  found  in  Stewart  and  Johnson  (1972),  Kloek  and  van  Dijk 
(1978),  Stewart  (1979),  van  Dijk  and  Kloek  (1980),  Zellner  and  Rossi  (1984),  Bauwens  and 
Richard  (1985),  Geweke  (1988),  and  Berger  and  Deely  (1988). 

Let  0  be  a  parameter  of  interest,  0€0  c  R*,  4(6)  be  the  likelihood  function  based  on  the 
observed  data  x  (an  n  -vector),  and  11(0)  be  a  prior  distribution.  The  posterior  distribution  is 
then 

where  5  (0)  =  (-<»,  0^'^x(-oo,  0(2)]x  x(-oo,  0^*^,  0^^  is  the  yth  component  of  6  and 
M,  =  J4(0)^n(0).  Let  ^  denote  the  collection  of  all  possible  actions  we  may  take  for  a  prob¬ 
lem  under  consideration  (e.g.,  iZ  =  9  in  the  problem  of  estimating  0).  d  is  assumed  to  be  a 
subset  of  .  Let  L  (0,  a  )^  be  the  loss  incurred  when  the  action  a  is  taken  and  0  is  the  true 
parameter.  A  Bayesian  solution  of  the  problem  is  an  action  a*  which  minimizes  the  posterior 
expected  loss 

r(a)  =  j£(0,a)dP,(0). 

Since  is  fixed  for  given  x,  a*  is  a  solution  of 

p(a*)  =  min^^^p(a), 

where 

p(a)  =  jL(0,a)4(0)dn(0). 

The  solution  a*  is  referred  to  as  Bayesian  action  in  the  literature.  Note  that  a*  may  not  be 
unique.  Only  in  special  cases  a*  can  be  obtained  analytically. 

The  numerical  approximation  to  a*  using  Monte  Carlo  method  is  obtained  as  follows. 

Select  a  distribution  H(B)  such  that  the  Radon-Nikodym  derivative  -4^(0)  exists  and  it  is  easy 

art 

to  generate  a  random  0  from  H.  Let  {  0;,  j  be  m  independent  and  identically 
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distributed  (i.i.d.)  random  it -vectors  generated  from  Approximate  p(a)  by 

P«(a)  =  fl)w(e,),  (1.1) 

fit  ‘  * 

where 

w(0)  =  /,(e)-^(e). 

The  Monte  Carlo  approximation  to  a*  is  an  action  satisfying 

This  approach  is  motivated  by  the  fact  that  for  any  fixed  a, 

lim«_U>m(a)  =  a)w(B)dH(e)  =  p(fl) 

for  almost  all  6^,  62,.-'  (with  respect  to  the  probability  distribution  H),  according  to  the  strong 
law  of  large  numbers  (SLLN). 

A  theoretical  justification  of  the  use  of  this  Monte  C^lo  method  is  the  convergence  of  the 
approximation  to  a  Bayesian  action  a*  in  some  sense.  In  some  simple  cases,  such  as 
L(B,  a)  is  the  squared  error  loss  (in  an  estimation  problem)  or  the  action  space  (Z  is  a  compact 
subset  of  ,  the  convergence  of  is  a  direct  consequence  of  the  SLLN  or  uniform  SLLN. 
Shao  (1988)  proved  the  almost  sure  convergence  of  in  the  situation  where  Cl  is  non¬ 
compact  but  the  loss  function  is  convex  in  a.  There  are  some  important  examples  of  convex 
loss  functions.  Also,  convex  loss  usually  ensures  the  uniqueness  of  the  Bayesian  action.  How¬ 
ever,  reasonable  loss  functions  derived  through  utility  analyses  are  often  not  convex  but 
bounded  and  concave  for  large  eiiors  (see  Berger,  1985,  Chapter  2).  For  a  convex  loss,  large 
errors  are  penalized  much  too  severely.  In  addition,  a  bounded  loss  function  usually  provides  a 
robust  Bayesian  solution  to  the  problem  (see  Section  2). 

In  this  note  we  study  the  limiting  behavior  of  the  Monte  Carlo  approximation  a„  for  gen¬ 
eral  unbounded  Cl  and  non-convex  loss  functions.  The  convergence  of  is  studied  in  Sec¬ 
tion  3  for  two  large  classes  of  loss  functions  introduced  in  Section  2.  The  rate  of  convergence 
and  the  asymptotic  distribution  of  (which  provides  an  accuracy  measure  for  the  Monte 
Carlo  approximation)  are  obtained  in  Section  4. 
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Throughout  the  paper  we  assume  that  x  is  a  fixed  data  vector,  0<m^<oo,  ^  is  a  closed 
subset  of  R^,  p(a)  is  finite  for  any  a^CL  and  a  Bayesian  action  a*  exists,  and  //  is  a  selected 
distribution  for  generating  random  6,- .  Discussions  for  the  selection  of  the  distribution  H  can 
be  found  in  Berger  (1985,  Section  4.9)  and  Geweke  (1988). 

2.  Preliminaries 

We  first  consider  the  modes  of  convergence  of  as  m  Let  co  denote  a  panic  ular 
sequence  (O^,  62>  • )  and  denote  the  corresponding  for  fixed  co.  Since  is  random, 
we  may  consider  the  almost  sure  convergence:  for  almost  all  to  (with  respect  to  H), 

n^(£D)->fl*.  (2.1) 

However,  unless  there  is  a  unique  Bayesian  action,  (2.1)  usually  does  not  hold.  For  practical 
uses,  a„  (co)  might  be  considered  as  a  gcxxl  approximation  as  long  as  a„  (co)  is  close  to  a  Baye¬ 
sian  action  a* .  That  is,  if 

=  C  a*:  a*  is  a  Bayesian  action  } 
and 

(l(a=  {  a:  a  is  a  limit  point  of  {  }  }, 

then  for  almost  all  co, 

a^<=a\  (2.2) 

Another  way  is  to  consider  the  posterior  expected  losses  r{a„)  and  r(a*),  since  the  pos¬ 
terior  expected  loss  is  used  to  judge  the  performance  of  an  action.  Note  that  r(a* )  is  uniquely 
defined  although  a*  may  not  Denote  r(a*)  and  p(a*)  by  r*  and  p*,  respectively.  It  is 
desired  to  show  that  for  almost  all  co, 

r(a^((0))  -4  r*, 

which  is  equivalent  to  (since  p(a)  =  Mjjr(a)) 

P(am(®)) P*-  (2.3) 

Usually  p(a)  is  continuous  in  a.  Then  (2.3)  is  weaker  than  (2.1). 
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The  following  result  relates  (2.2)  to  (2.3)  and  the  boundedness  of  a^(to): 

II  (CO)  II  !SC{o  for  all  m,  (2.4) 

where  Co)>0  is  a  constant  for  each  o),  \\a  II  =  (a^a)'^*  and  is  the  transpose  of  a. 

Lemma  1.  Let  (O  be  fixed.  Suppose  that  p(a)  is  continuous  and  that 

>  P*-  (2.5) 

Then  (2.3)  is  equivalent  to  (2.2)  and  (2.4). 

Proof.  Suppose  that  (2.3)  holds.  If  (2.4)  does  not  hold,  then  there  is  a  subsequence 
{  fl„/to),  7=1,2,... }  such  that 

limy^lla„^(CD)ll  =oo. 

From  condition  (2.S),  this  implies 

lim^-4«p(<*m/®))  >  P*. 

which  contradicts  (2.3).  Hence  (2.4)  holds.  Let  ced^^.  Then  there  is  a  subsequence 
{  a^^(<o),  /=1,2,... }  such  that  lim/_^;^((0)  =  c.  From  the  continuity  of  p, 

liro/-»-P(am,(©))  =  P(«^)- 

From  (2.3),  p(c)  =  p*.  Hence  ce<2*  and  therefore  (2.2)  holds. 

Suppose  now  (2.2)  and  (2.4)  hold.  Let  q  be  any  limit  point  of  {  p{a„{(£i)),  m=\,2,... }. 
Then  there  is  a  subsequence  {  ntj  }  such  that 

lim^_»o4>(a«/CD))  =  q. 

From  (2.2)  and  (2.4),  there  is  a  subsequence  {  nti  }c:{  mj  }  such  that 

lim/_^fl^(0})  =  a*  e  a*. 

From  the  continuity  of  p,  q  =  p* .  This  proves  (2.3). 

Berger  (1985)  pointed  out  that  reasonable  loss  functions  are  usually  bounded.  Consider 
loss  functions  satisfying  the  following  condition; 
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Condition  (LI). 

(1)  L(6,  a)  is  continuous  in  a  and  supe.a^(0.  a)  <  «». 

(2)  For  any  constant  C >0,  limn^  i_4.X(0.  a)  =  uniformly  for  all  0  satisfying 
II 6 II  ^  C ,  where  A  is  a  fixed  constant. 

A  simple  example  of  a  loss  function  satisfying  (LI)  is 

L  (0,  a )  =  min(  II  0-a  II  A ), 

where  A  is  a  constant.  Note  that  A  usually  is  an  upper  bound  of  the  loss  function.  Hence  for  a 
reasonable  loss  function  satisfying  (LI),  it  is  usually  true  that  A  >  r* . 

The  following  result  shows  condition  (LI)  implies  the  conditions  in  Lemma  1 . 

Lemma  2.  Assume  (LI).  Then  p(a)  is  continuous  and  (2.5)  holds  if  A  >r* . 

Proof-  It  is  obvious  that  the  continuity  and  boundedness  of  L  imply  the  continuity  of  p.  For 
(2,5),  it  suffices  to  show  that 

liroiia !_»«''(«)  =  (2.6) 

For  any  e>0,  since  =  J/,(0)dn(0)  is  finite,  there  exists  a  constant  C  >0  such  that 

<  e. 

For  this  C  >0,  under  condition  (LI),  there  exists  &K>0  such  that  when  II a  II  >  K, 

li'Ce,  a)  -  A  I4(0)dn(0)<  e. 

Hence  for  II  a  II  >  K, 

flL(0,  a)  -  A  l/j,(0)dn(0)  <  (A+B+l)e, 
where  B  =  sup  g  (0,  a ).  This  proves  (2.6)  since  e  is  arbitrary, 

It  can  be  shown  that  when  the  loss  function  satisfies  (LI)  and  the  Bayesian  action  a*  is 
unique,  the  Bayesian  action  is  robust  in  the  sense  that  for  any  sequence  of  posteriors  {  G„  } 
converging  weakly  to  P^,  we  have  a„  a*,  where  a^  is  a  Bayesian  action  corresponding  to 
,  We  will  not  discuss  this  issue  here. 
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An  unbounded  loss  function  usually  satisfies  liniiia ||^«L(0,  a)  =  oo  for  fixed  6.  We  con¬ 
sider  the  loss  functions  satisfying  the  following  condition. 

Condition  (L2). 

(1)  L  (0,  a )  is  continuous  in  a  and  for  any  C  >0,  there  is  a  function  Bq  (0)  such  that 
sup  Iia  Iisc^(0.  ^  ^c(®)  and  |Bc(0)4(®)«^n(0)  <  oo. 

(2)  There  is  a  constant  Cq  such  that  dPxi^)  >  0  and  lim|u||_^(0,  a)  =  «» 

holds  uniformly  for  0  satisfying  11011  S  Cq. 


From  the  dominated  convergence  theorem,  the  first  condition  in  (L2)  implies  the  con¬ 
tinuity  of  p(a).  The  second  condition  in  (L2)  implies  (2.5),  as  the  following  lemma  shows. 


Lemma  3.  Assume  the  second  condition  in  (LI).  Then 


and  therefore  (2.5)  holds. 

For  any  K  >  0,  there  is  a  Ari>  0  such  that  for  any  a  with  Ha  II  >  ATi, 

inf  ii0iisCo^(®* 


Then 

The  result  follows  since  K  is  arbitrary  and  J  „  ^  >  0. 

11  0  II  A 


We  also  need  the  following  technical  lemma  for  the  proof  of  the  main  results. 


l-^ma_4.  Let  g(0,  a)  be  a  function  on  R*xR^  and  F  be  a  distribution  function  on  R*.  Sup¬ 
pose  that  for  fixed  a ,  g  is  measurable  and  for  fixed  0,  g  is  continuous.  Suppose  also  that  for 
any  C  >  0,  there  is  a  function  BciQ)  such  that  sup  lg(0,  a)l  $5^(0)  and 
J5c(0)ttf"(0)  <  oo.  Let  0i,...,0„,  be  i.i.d.  samples  from  F.  Then  for  almost  all  0i,02,..., 
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sup  1  ~  I? (0.  <»)^(6) I  -♦  0  for  all  positive  rational  C.  (2.7) 

Proof-  For  any  fixed  C ,  from  Theorem  2  of  Jennrich  (1969), 

Ha  use  '  .  a )  -  /« (0.  (0)  I  -^0 

holds  for  almost  all  61,82,...  Then  (2.7)  follows  from  the  fact  that  the  set  of  all  rational 
numbers  is  countable. 

3.  Convergence  of  Monte  Carlo  approximations 

For  loss  functions  satisfying  either  condition  (LI)  or  (L2),  the  convergence  of  (in  the 
sense  of  (2.2)  and  (2.3))  is  established  in  the  following  theorems. 

Theorem  1.  Assume  that  L(6.  a)  satisfies  condition  (LI)  with  A  >  r* .  Then  (2.2)  and  (2.3) 
hold  for  almost  aU  (O  (with  respect  to  //). 

Proof.  From  Lemma  2,  the  conditions  of  Lemma  1  are  satisfied.  Using  Lemma  1,  we  only 
need  to  show  (2.2)  and  (2.4). 

Consider  (2.4)  first  Note  that  /w(8)ctf/(6)  = /4(0)i/n(8)  =  <  «».  From  the  SLLN, 

for  almost  all  m, 

(0i  ( 1 0,  II  >c  )  ^  J,  0 „ ^  ^  (0)  for  all  positive  rational  C ,  (3.1) 

where  Ig  is  the  indicator  function  of  the  set  S.  Let  a*  be  a  Bayesian  action.  From  (1.1)  and 
the  SLLN,  for  almost  all  ©, 

(3.2) 

Let  (0  be  fixed  such  that  (3.1)  and  (3.2)  hold.  Suppose  that  (2.4)  does  not  hold.  Then  there  is  a 
subsequence  of  diverging  to  infinity.  Without  loss  of  generality,  assume  that 

II  (to)  II  — >  <».  For  any  e>0,  there  is  a  rational  C  >0  such  that 

f  w(e)d//(0)  <  e. 

''«eii>c  ^ 

From  condition  (LI),  for  this  C  >0,  there  is  an  N^>0  such  that  for  all  II 6 II  ^  C  and  m  > 

\L(,e,  an((a))  -  A  I  <  e. 
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Then 


^sr..  ii.(e„  a„((0))  -  A  Iw(6,)  s  w,ii>c  )• 


m 


m 


where  B  »  supe,aL(0,  a).  From  (3.1), 


1 


limsup ^»n(<«>»  -  A.  Iw(e,)  ^  (i4+fl+M,)e 

tn  *“* 


and  therefore 


P«(am(o>))-»M,A.  (3.3) 

But  from  (1.2), 

Pm(a«(0»)  ^  Pm(a*) 

for  all  m .  Hence  from  (3.2), 

linisup^_^«(a„(©))  S  p*. 

This  contradicts  (3.3)  since  m^A  >  p* .  Hence  (2.4)  holds  for  almost  all  co. 

From  Lemma  4,  for  almost  all  co, 

sup  1^11^  lp«(a)  -  p(a)l  -»  0  for  all  positive  rational  C.  (3.4) 

Let  0)  be  fixed  such  that  (2.4)  and  (3.4)  hold  and  (3.2)  holds  for  an  a* efl* .  Then 

I  (a„  (CO))  -  p(a„ (to))  I  ->  0.  (3.5) 

Let  a  i€  C2jo.  Then  II  a  j  II  and  there  is  a  subsequence  {  ntj  }  such  that 

lim^_M.flmy(<o)  =  fli- 

From  the  continuity  of  p  and  (3.5),  we  have 

Umy_^^^(a^/to))  =  p(ai). 

But 

Pm/a»,/©))  ^  Pm/fl*). 

which  converges  to  p*  by  (3.2).  Hence  p(a,)  =  p*  and  Oiea*.  This  proves  (2.2).  ° 


Theorem  2.  Assume  that  L(B,  a)  satisfies  condition  (L2).  Then  (2.2)  and  (2.3)  hold  for  almost 
all  (0. 
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Proof»  From  Lemma  3,  the  conditions  of  Lemma  1  are  satisfied  and  therefore  we  only  need  to 
show  (2.2)  and  (2.4).  Note  that  for  almost  all  to. 

For  a  fixed  co  such  that  (3.2)  and  (3.6)  hold,  we  have 

(“))  ^  liminf  (“))w  (Q/ y ( no,  IlsCo  ) 

^  Uminf  ,„^(inf  ,i0a5Co^(e.  am(to))-^5:,^,w(e,)/(  i,e,iKCo)] 

^  liminf  ^_^inf  iieii5Co^(0.  a;n(®))J||eiisc„'^(0)^(9) 
and 

limsup„_«pm(^m(®))  ^  =  P*  • 

From  condition  (L2),  limujn  iieiisc/'C®*  ^)  =  *“•  Hence  {  a„(to) }  can  not  have  any 
subsequence  diverging  to  infinity.  Therefore  (2.4)  holds  for  almost  all  co. 

From  Lemma  4,  (3.4)  holds  under  condition  (L2).  Then  the  prtwf  of  (2.2)  is  the  same  as 
that  in  the  prcwf  of  Theorem  1.  This  completes  the  proof  of  Theorem  2. 

From  (2.2),  if  the  Bayesian  action  is  unique,  then  a„  converges  to  a*  in  the  ordinary 
sense. 

Corollary  1.  Assume  the  conditions  of  Theorem  1  or  Theorem  2.  If  the  Bayesian  action  is 
unique,  then  (2.1)  holds  for  almost  all  co. 

In  estimation  problems,  it  is  desired  to  indicate  the  accuracy  of  the  Bayes  estimate  a  * . 
The  posterior  expected  loss  of  a*,  of  course,  can  be  used  as  an  accuracy  measure.  Let 

can  be  used  to  approximate  the  posterior  expected  loss  r* . 
The  following  result  is  a  direct  consequence  of  (3.5)  and  Theorem  1  (or  Theorem  2). 

Corollary  2.  Assume  the  conditions  of  Theorem  1  or  Theorem  2.  Then  for  almost  all  to, 
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4.  Convergence  rate  and  limiting  distribution 

We  study  the  convergence  rates  of  a^(a})  and  r(a„(a}))  for  differentiable  loss  functions. 
Let  VL(0,  a)  =  3L(0,  a)/da,  V^L(0,  a)  =  d^L(d,  a)/dada^  and  fl)  be  the  (M,v)th  ele¬ 
ment  of  V^L  (0,  a ),  1^ ,  V  ^ . 

Condition  (L3). 

(1)  For  almost  all  0  (with  respect  to  H),  V^L(0,  a )  exists  for  any  a  and  is  continuous 
in  a. 

(2) JllVL(0,a*)llV(0)dW(0)  <  ooforanya'ea*. 

(3)  For  any  C  >  0,  there  is  a  function  Bc(B)  such  that  jBc(0)lx(0)<^n(0)  <  <»  and 

^Bc(0)foraU  iSu.v^. 

(4)  For  any  a*eCl*.  Jv^L(0,  a*)Zj(0)dn(0)  is  positive  definite. 

Under  condition  (L3),  p(a)  is  second  order  continuously  differentiable  and  Vp(a*)  =  0 
for  any  a*eC[*,  where  Vp  is  the  gradient  of  p. 

If  the  Bayesian  action  is  unique,  a„(0})  converges  to  a*  for  almost  all  (o  (Corollary  1) 
and  the  convergence  rates  of  a„(<i))  and  r(a„(a)))  can  be  obtained  by  using  standard  tech¬ 
niques.  Shao  (1988)  obtained  the  convergence  rates  when  the  loss  is  convex.  The  result  is 
extended  to  general  non-convex  loss  situations  (see  Theorem  4  below).  When  the  loss  function 
is  not  convex,  the  Bayesian  action  may  not  be  unique.  If  there  are  more  than  one  Bayesian 
actions,  a;„(0))  may  not  converge  in  the  ordinary  sense  and  it  is  also  much  more  difficult  to 
obtain  the  convergence  rate  of  r(a^(©))  (although  r(a„(0i>))  converges  according  to  Theorems 
1  and  2).  In  Theorem  3,  without  assuming  the  uniqueness  of  the  Bayesian  action,  we  establish 
a  convergence  rate  for  r  (a;„  (to))  in  some  situations. 

Theorem,,  3.  Assume  (L3)  and  either  (LI)  or  (L2).  Assume  also  that  for  almost  all  ca, 

XHi ,  a*  )w(Bi)  =  0  (m '^’(loglog m  )'^*)  for  all  a  *  €  3(2  * ,  (4.1) 

where  dd*  is  the  boundary  of  d*.  Then  for  almost  all  O), 
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r(a„i(0))  -  r*  =  o(m"'/*(loglogm)'^’). 

Remark.  Note  that  for  any  a*ed(Z*,  £*jVL(0,-,  a*)w(di)  is  a  sum  of  random  variables 
and  Vp(a*)  =  JVL(0,  a*)w(0)dW(0)  =  0.  Hence  from  condition  (L3)  and  the  law  of  iterated 
logarithm,  for  almost  all  m, 

5:;^jVL(0;,  a*)w(0.)  =  O(m'/*aoglogm )'/*). 

Thus,  condition  (4.1)  is  clearly  satisfied  if  d^*  is  a  countable  set.  An  important  example  of 
countable  Bd*  is  that  ^  c  R  and  d*  =  where  are  constants  and  A  is  a 

countable  index  set.  Another  example  is  that  is  a  countable  set  (dd*c:d*  since  d*  is 
closed). 

Proof.  Let  Q(a)  =  Jv^L(0,  a)w(fi)dH (0).  Note  that  under  either  (LI)  or  (L2),  (2.2)  and  (2.4) 
hold  (Theorems  1  and  2).  Also,  from  condition  (L3)  and  Lemma  4, 

Ha  use  *  ®  for  all  positive  rational  C.  (4.2) 


Let  (0  be  fixed  such  that  (2.2),  (2.4)  and  (4.1)-(4.2)  hold,  and 

...  m’^[p(a«(CD))-p*] 

^  .1/, 

Ooglogm)'* 

It  suffices  to  show  that  for  any  subsequence  {mi  },  there  is  a  subsequence  {  mj  }  a  {  mi  } 
such  that 

limy_^„/©)  =  0. 


Let  {mi  }  be  a  given  subsequence.  From  (2.4),  there  is  a  subsequence  {  mj  }(z{  mi  }  such 
that 


liniy_^„^(co)  =  a*ea*. 


(4.3) 


Case  1.  a*Bd*~dd* .  Since  a*  is  an  interior  point  of  d*,  there  is  a  constant  5>0  such  that 
p(a)  =  p*  or  all  a  satisfying  lla-a*  II  <5.  Then  from  (4.3), 

Zmy(®)  =  0  for  sufficiently  large  j. 

Case  2.  a*eBd* .  Note  that  £,^^jVL(0f ,  a„j{<o))w{Qi)  =  my  Vp„^(a;„^(co))  =  0.  Then  from  the 
mean  value  theorem. 


II 


SjjVLO,-.  a*)w(e,.)  =  ^y(Q)))w(e.)][a*^^/a))]  (4.4) 

and 

P(am/CD))  -  p*  =  [Vp(C;(a)))r[fl„^(co)-fl*],  (4.5) 

where  (©)  and  C;(<o)  are  on  the  line  segment  between  a*  and  a^^isa).  From  (2.4)  and  (4.2), 

(4.6) 

ntj 

which  is  positive  definite  under  (L3).  Since  a*edCl* ,  (4.1)  holds  and 

limj_»«Vp(Cj((D))  =  Vp(fl*)  =  0. 

Hence  limy_^;„^(©)  =  0  follows  from  (4.1)  and  (4.3)-(4.6).  This  completes  the  proof  since 
r(a)  =  p(a)/M;j.  O 

Theorem  4.  Assume  the  same  conditions  as  in  Tlieorem  3  and  there  is  a  unique  Bayesian  action 
a  * .  Then  for  almost  all  cn, 

(©)  -  a  *  -O  (w"'^’(loglog  m  )'^*),  (4.7) 

and 

ria„((0))  -  r*  =  0(/n~hoglogm).  (4.8) 

By  the  same  argument  as  in  the  proof  of  Theorem  3,  we  can  show  that  for  almost  all  co, 

— 2:i'!iV2L(0.-,  6«)w(0i)  -4  /v2l(0,  a*)>v(0)<tf/(0)  >  0.  (4.9) 

where  {  bm  }  is  any  sequence  satisfying  \\b„-a*  llSlla^(a))-o*  II.  From  the  mean  value 
theorem  and  the  fact  that  Vp(a* )  =  0  and  £"jVL(0,  ,  a;„(©))H'(0,  )  =  0,  we  have 

5:“jVL(0,.,  a*)w(0,)  =  ^(o)))](a*^«(a))]  (4.10) 

and 

p(am  (®))  -  P*  =  [flm  *  l^V^PCCm  (®)-« *  1.  (4. 1 1 ) 

where  ^(o))  and  Cm(®)  are  on  the  line  segment  between  a*  and  a^io)).  Then  (4.7)  follows 
from  (4.9)-(4.10)  and  the  law  of  iterated  logarithm  and  (4.8)  follows  from  (4.7),  (4.9)  and 
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(4.11).  ° 

From  (4.7)-(4.8),  r(a„((o))  converges  much  faster  than  anifo).  In  the  following  we 
obtain  a  limiting  distribution  of  a„ ,  which  provides  an  accuracy  measure  for  Om- 

Theorem  5.  Assume  the  same  conditions  as  in  Theorem  4.  Then 

m -  a*)  N(P,D)  in  distribution , 

where  N  (0,0)  is  the  p  dimensional  normal  distribution  with 

D  =  U'Wu-\ 

V  =  J[VL(0,  a*))[VL(e,  a*)]^wHe)dH(6), 

U  =  Jv^LCe,  a*)w(Q)dH(Q). 

Proof.  The  result  follows  from  (4.9)-(4.10)  and  the  central  limit  theorem.  ^ 

A  Monte  Carlo  approximation  to  Z>  is 

D„(©)  =  U-\(i>)V^((0)U-\(0) 

with 

V„(fo)  =  — S«.[VL(0.-.  a„(©))][VL(0i,  fl«(©))]V2(0,.) 

fn 

and 

m 

Using  the  same  argument  as  in  the  above  proofs,  we  can  show  the  following  result 

Theorem  6.  Assume  the  same  conditions  as  in  Theorem  4.  If  for  any  C  >  0,  there  is  a  func¬ 
tion  Bc(0)  such  that  jBc(0)w^(0)<tf/(0)  <  oo  and  sup  ^  ®c(0)*  for 

almost  all  co, 

=  D . 


13 


References 


Bauwens,  W.  and  Richard,  J.  F.  (1985).  A  1-1  plot-t  random  variable  generator  with  applica¬ 
tion  to  Monte  Carlo  integration.  J.  Econometrics  29,  19-46. 

Berger,  J.  O.  (1985).  Statistical  Decision  Theory  and  Bayesian  Analysis,  second  edition. 
Spring- VCTlag,  New  York. 

Berger,  J.  O.  and  Deely,  J.  (1988).  A  Bayesian  approach  to  ranking  and  selection  of  related 
means  with  alternatives  to  analysis-of-variance  methodology.  J.  Amer.  Statist.  Assco. 
83,  364-373. 

Geweke,  J.  (1988).  Bayesian  inference  in  econometric  models  using  Monte  Carlo  integration. 
Econometrica,  to  appear. 

Hanunersley,  J.  M.  and  Handscomb,  D.  C.  (1964).  Monte  Carlo  Methods.  Methuen,  London. 
Jennrich,  R.  I.  (1969).  Asymptotic  properties  of  non-linear  least  squares  estimators.  Ann.  Math. 
Statist.  40,  633-643. 

Kloek,  T.  and  van  Dijk,  H.  K.  (1978).  Bayesian  estimates  of  equation  system  parameters;  an 
application  of  integration  by  Monte  Carlo.  Econometrica  46,  1-19. 

Shao,  J.  (1988).  Monte  Carlo  approximations  in  Bayesian  decision  theory.  Technical  Report, 
Department  of  Statistics,  Fhirdue  University. 

Stewart,  L.  (1979).  Multiparameter  univariate  Bayesian  analysis.  J.  Amer.  Statist.  Assoc.  74, 
684-693. 

Stewart,  L.  and  Johnson,  J.  D.  (1972).  Determining  optimum  bum-in  and  replacement  times 
using  Bayesian  decision  theory.  IEEE  Transactions  on  Reliability  R-2I  170-175. 
van  Dijk,  H.  K.  and  Kloek,  T.  (1980).  Further  experience  in  Bayesian  analysis  using  Monte 
Carlo  integration.  J.  Econometrics  14,  307-328. 

Zellner,  A.  and  Rossi,  P.  (1984).  Bayesian  analysis  of  dichotomous  quantal  response  models. 

/.  Econometrics  25,  365-394. 


14 


REPORT  DOCUMENTATION  PAGE 


DCOASSmCATION/OOWNGRAOING  SCHEDULE 


4.  FERFORMING  ORGANIZATION  REPORT  NUMRER(S) 


MARKING 


OISTRIIUTION/AVAILAIIUTY  Of  REPORT 

Approved  for  public  release,  distribution 
uni  a...  "ted. 


S.  MONITORING  ORGANIZATION  REPORT  NUMSER($) 


Technical  Report  #88-65C 


6a.  NAME  Of  PERfORMING  ORGANIZATION 


6b.  Of fia  SYMBOL  I  7a.  NAME  Of  MONITORING  ORGANIZATION 
OfsppIkMbh)  I 


7b.  ADDRESS  (Oiy,  State,  snd  ZIP  Cock) 


6b.  OfflCE  SYMBOL  1 9.  PROCUREMENT  INSTRUMENT  lOENTIfICATION  NUMBER 


Of  appHetbk) 


Purdue  University 


6c  ADDRESS  (OQt.  State,  and  ZlPCodt) 

Department  of  Statistics 
West  Lafayette,  IN  47907 


Ba.  NAME  Of  fUNDING /SPONSORING 
ORGANIZATION 

Office  of  Naval  Research 


Be  ADDRESS  (CRy.  State,  and  ZIPCodt) 

Arlington,  VA  22217-5000 


11.  TITLE  Qnduda  Saeurity  Ossafkatlon/ 

MONTE  CARLO  APPROXIMATIONS  IN  BAYESIAN  DECISKMl  THEORY  PART  III;  LIMITING  BEHAVIOR  OF 


NSF  DMS-8717799.  DMS-8702620 
N00014-88-K-1Q70 


10.  SOURCE  OF  FUNDING  NUMBERS 


PROGRAM  iPROJEa 
ELEMENT  NO.  I  NO. 


12.  PERSONAL  AUTHOR(S) 


13a.  TYPE  Of  REPORT 


16.  SUPPUMENTARY  NOTATION 


13b.  TIME  COVERED 
FROM _ _  TO 


14.  DATE  OF  REPORT  (ftar.  Month.  Day)  hS.  PAGE  COUNT 


17. _  COSATi  COOES  I  16.  SUBJECT  TERMS  (Condnue  on  nveaa  If  nantsary  and  Idantify  by  bloak  numbar) 

FIELD  I  GROUP  |  SUB-GROUP  |  Almost  sure  convergence;  convergence  rate;  loss  function; 

posterior  expected  loss. 


19.  abstract  (Cbnt/nue  on  ravana  If  nacastary  and  idantify  by  Modi  numbar) 

Monte  Carlo  approximation  is  a  useful  method  in  obtaining  a  numerical  approximation 
to  a  Bayesicui  action  (an  action  which  minimizes  the  posterior  expected  loss) .  We  study 
the  behavior  of  the  Monte  Carlo  approximation  when  the  Monte  Carlo  sanple  size  is  large. 
Convergence  euid  convergence  rate  of  the  Monte  Carlo  approximation  are  estcLblished  under 
some  weak  conditions  on  the  loss  function. 


20.  DISTRIBUTION /AVAILABIUTY  OF  ABSTRACT  1 21.  ABSTRAa  SECURITY  CLASSIFICATION 

OUNCLASSIFIEO/UNLIMITEO  03  SAME  AS  RPT.  □  OTIC  USERS  I  Unclassified 


22a.  NAME  OF  RESPONSIBLE  INDIVIDUAL  |22b.  TELEPHONE  (Jne/wdr  AraaCoda)  22c.  OFFICE  SYMBOL 

Jun  Shao  I  317-494-6039 


00  FORM  1473,  B4  MAR  B3  APR  edition  may  b«  u»ad  until  tahaurted.  SECURITY  CLASSIFICATION  OF  "HIS  PAGE 

All  other  editiont  are  obwlete. 

UNCLASSIFIED 


